Method and device for determining distance

ABSTRACT

A method and device for determining a distance are provided. The method includes determining a first binocular distance between two eyes of a user by using a first image captured through a camera of the device, determining a second binocular distance between the two eyes of the user by using a second image captured through the camera, determining a distance from the camera to the two eyes of the user based on the first binocular distance and the second binocular distance, and providing an image to a display of the device, the image converted according to the distance from the camera to the two eyes of the user.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit under 35 U.S.C. § 119(e) of a U.S.Provisional application filed on Nov. 14, 2016, in the U.S. Patent andTrademark Office and assigned Ser. No. 62/421,682, and under 35 U.S.C. §119(a) of a Korean patent application filed on Mar. 10, 2017, in theKorean Intellectual Property Office and assigned Serial No10-2017-0030533, the entire disclosure of each of which is herebyincorporated by reference.

TECHNICAL FIELD

The present disclosure relates to a method and a device for determininga distance. More particularly, the present disclosure relates to amethod and a device for determining a distance by using a camera.

BACKGROUND

Various methods are used to measure a distance to a subject. Forexample, a method of using light in order to measure a distance to asubject is widely used. Examples of a method of measuring a distance byusing light include a triangulation method, a time-of-flight (TOF)method, and a phase-shift method.

The triangulation method is a method of measuring a distance by usingtriangulation, and the TOF method is a method of calculating a distanceby using a difference between a time when light is emitted from adistance measuring apparatus and a time when the light returns to thedistance measuring apparatus after being reflected from a subject. Thephase-shift method is a method of calculating a distance based onphase-shift between reference light and light that has returned to adistance measuring apparatus after being reflected from a subject due toirradiation of light having a uniform frequency to the subject.

Alternatively, a method of measuring a distance to a subject by using aplurality of images obtained through a plurality of cameras may be used.For example, stereo matching using a stereo camera is a process ofrecognizing a 3-dimensional (3D) structure by using two eyes of aperson, which is realized in terms of hardware, and is a method ofextracting information about a depth (or a distance) in a space througha process of analyzing a pair of images obtained by photographing onesubject using two cameras. By calculating a value obtained from an inputimage, 3D distance information of an observation space may be measured.

However, according to such general technologies, complicated hardwarestructures are required while irradiating and receiving special light orusing a plurality of cameras.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

Aspects of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentdisclosure is to provide a method and a device for determining adistance through two or more photographing operations.

Another aspect of the present disclosure is to provide a method and adevice for obtaining sensed information from a sensor and determining adistance by using the sensed information.

In accordance with an aspect of the present disclosure, a method bywhich a device determines a distance is provided. The method includesdetermining a first binocular distance between two eyes of a user byusing a first image captured through a camera of the device, determininga second binocular distance between the two eyes of the user by using asecond image captured through the camera, determining a distance fromthe camera to the two eyes of the user based on the first binoculardistance and the second binocular distance, and providing an image to adisplay of the device, the image converted according to the distancefrom the camera to the two eyes of the user.

In accordance to another aspect of the present disclosure, a device fordetermining a distance is provided. The device includes a cameraconfigured to photograph two eyes of a user, and a processor configuredto determine a first binocular distance between the two eyes of the userby using a first image captured through the camera, determine a secondbinocular distance between the two eyes of the user by using a secondimage captured through the camera, determine a distance from the camerato the two eyes of the user based on the first binocular distance andthe second binocular distance, and provide an image to a display of thedevice, the image converted according to the distance from the camera tothe two eyes of the user.

In accordance with another aspect of the present disclosure, anon-transitory computer-readable recording medium has recorded thereon aprogram which, when executed by a computer, performs the method.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a diagram for describing an example in which a devicedetermines a distance from the device to a user, according to anembodiment of the present disclosure;

FIG. 2 is a flowchart of a method by which a device determines adistance from the device to a user, according to an embodiment of thepresent disclosure;

FIG. 3 is a diagram for describing an example in which a devicedetermines a distance from the device to a user by using a firstbinocular distance and a second binocular distance, according to anembodiment of the present disclosure;

FIG. 4 is a flowchart of a method by which a device determines adistance from the device to a user by using sensing information,according to an embodiment of the present disclosure;

FIG. 5 is a diagram for describing an example in which a devicedetermines a distance from the device to a user by using a firstbinocular distance, a second binocular distance, and a sensing unit,according to an embodiment of the present disclosure;

FIG. 6 is a diagram for describing an example in which a devicedetermines a distance from the device to a user by using an eyeball sizeof the user, according to an embodiment of the present disclosure;

FIG. 7 is a flowchart of a method by which a device determines adistance from the device to a user by using an eyeball size of the user,according to an embodiment of the present disclosure;

FIG. 8 is a diagram for describing an example in which a devicedetermines a distance from the device to a user by using a predictedeyeball size, according to an embodiment of the present disclosure;

FIG. 9 is a diagram for describing an example in which a device uses aguideline to determine a distance, according to an embodiment of thepresent disclosure;

FIG. 10 is a flowchart of a method by which a device determines adistance from the device to a user by using a degree of tilt of thedevice, according to an embodiment of the present disclosure;

FIG. 11 is a diagram for describing an example in which a devicedetermines a distance from the device to a user when a camera is locatedhigher than a height of two eyes of the user according to an embodimentof the present disclosure;

FIG. 12 is a diagram for describing an example in which a devicedetermines a distance from the device to a user when a camera is locatedlower than a height of two eyes of the user according to an embodimentof the present disclosure;

FIG. 13 is a flowchart of a method by which a device processes an imagebased on a distance from the device to a user and user information,according to an embodiment of the present disclosure;

FIG. 14 is a diagram for describing an example in which a devicedetermines a location of a focal plane of an image based on a distancefrom the device to a user and user information, according to anembodiment of the present disclosure;

FIG. 15 is a diagram for describing an example in which a deviceprovides an image to a user by using a pinhole mask, according to anembodiment of the present disclosure;

FIG. 16 is a flowchart of a method by which a device processes an imageaccording to a type of content being displayed, according to anembodiment of the present disclosure;

FIG. 17 is a flowchart of a method by which a device processes an imagebased on whether a type of content being displayed is video or text,according to an embodiment of the present disclosure;

FIG. 18 is a diagram for describing an example in which a devicedetermines a distance between the device and a user, and displays thedetermined distance, according to an embodiment of the presentdisclosure;

FIG. 19 is a diagram for describing an example in which a deviceprovides a guiding instruction to a user in order to photograph two eyesof the user, according to an embodiment of the present disclosure; and

FIGS. 20 and 21 are block diagrams of devices according to variousembodiments of the present disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the present disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thepresent disclosure. In addition, descriptions of well-known functionsand constructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of the presentdisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of the presentdisclosure is provided for illustration purpose only and not for thepurpose of limiting the present disclosure as defined by the appendedclaims and their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

In the specification, when a region is “connected” to another region,the regions may not only be “directly connected,” but may also be“electrically connected” via another device therebetween. Also, when aregion “includes” an element, the region may further include anotherelement instead of excluding the other element, otherwise differentlystated.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list.

Hereinafter, examples of the present inventive concept will be describedwith reference to accompanying drawings.

FIG. 1 is a diagram for describing an example in which a devicedetermines a distance from the device to a user, according to anembodiment of the present disclosure. For example, the device maydetermine a distance from a lens of a camera included in the device totwo eyes of the user.

Referring to FIG. 1, a device 100 according to an embodiment maydetermine a distance by using one camera. For example, the device 100may determine the distance from the lens of the camera to the two eyesof the user through two or more photographing operations using onecamera included in the device 100. The device 100 may determine thedistance from the lens of the camera to the two eyes of the user bycomparing information about the two eyes of the user obtained throughfirst photographing and second photographing.

For example, the device 100 may determine a first binocular distancebetween the two eyes of the user by using a first image captured by thecamera, determine a second binocular distance between the two eyes ofthe user by using a second image captured by the camera, and determinethe distance from the camera to the two eyes of the user based on thefirst and second binocular distances. For example, the device 100 maydetermine the distance from the camera to the two eyes of the user bycomparing the first binocular distance that is the distance between thetwo eyes of the user with the second binocular distance that is thedistance between the two eyes of the user, wherein the first binoculardistance and the second binocular distance are recognized by an imagesensor of the camera when the first image and the second image arecaptured, respectively. As another example, the device 100 may determinethe distance from the camera to the two eyes of the user by comparingthe first binocular distance that is the distance between the two eyesof the user in a focal plane of the camera when the first image iscaptured, with the second binocular distance that is the distancebetween the two eyes of the user in the focal plane of the camera whenthe second image is captured.

The device 100 according to an embodiment may provide an image convertedaccording to the distance from the camera to the two eyes of the user,to a display of the device 100. For example, the device 100 may processthe image to be displayed on the display based on the distance from thecamera to the two eyes of the user. The device 100 may use userinformation while processing the image. For example, the device 100 mayprocess the image to be displayed on the display by using the distancefrom the camera to the two eyes of the user and visual acuityinformation of the user. An example thereof will be described later withreference to FIG. 13.

The device 100 according to an embodiment may obtain sensed informationfrom a sensor, and determine a distance by using the sensed information.For example, the device 100 may obtain two images of the user throughtwo or more photographing operations, obtain motion information of thedevice 100 from the sensor, and analyze a difference between the twoobtained images based on the motion information, thereby determining thedistance from the user to the device 100. For example, the device 100may determine the distance from the lens of the camera to the two eyesof the user by using a distance between the two eyes of the userobtained through first photographing, a distance between the two eyes ofthe user obtained through second photographing, and a moving distance ofthe device 100 between the first photographing and the secondphotographing.

The device 100 according to an embodiment may determine an eyeball sizeof the user by photographing an eyeball of the user, predicting theeyeball size of the user by using the user information, and determiningthe distance between the device 100 and the user based on the determinedeyeball size and the predicted eyeball size. The device 100 maydetermine the distance from the lens of the camera to the two eyes ofthe user by comparing the determined eyeball size and the predictedeyeball size.

The device 100 according to an embodiment may determine the distancebetween the device 100 and the user, and process the image to bedisplayed by using the determined distance. The device 100 may processthe image to be displayed based on the visual acuity information of theuser stored in the device 100, and the distance from the lens of thecamera to the two eyes of the user. For example, the device 100 maydetermine a location of a focal plane of the image to be displayed,based on the visual acuity of the user and the distance from the lens ofthe camera to the two eyes of the user. For example, when the user isnearsighted, the device 100 may determine the location of the focalplane of the image to be displayed to be closer to the two eyes of theuser than a location of the display. As another example, when the useris farsighted, the device 100 may determine the location of the focalplane of the image to be displayed to be farther from the two eyes ofthe user than the location of the display. As another example, thedevice 100 may determine directivity of light displayed on the displayaccording to the visual acuity of the user.

FIG. 2 is a flowchart of a method by which a device determines thedistance from the device to the user, according to an embodiment of thepresent disclosure. For example, the device may determine the distancefrom the lens of the camera to the two eyes of the user by using onecamera.

Referring to FIG. 2, the distance from the device 100 to the user maydenote a distance from a pre-set portion of the device 100 to a pre-setbody part of the user. For example, the distance from the device 100 tothe user may denote the distance from the lens of the camera to the twoeyes of the user. As another example, the distance from the device 100to the user may denote a distance from the display included in thedevice 100 to the two eyes of the user. As another example, the distancefrom the device 100 to the user may denote a distance from a center ofthe display included in the device 100 to a portion of a face of theuser, the portion of the face being closest to the display. However,examples of the distance from the device 100 to the user are not limitedthereto.

In operation S210, the device 100 according to some embodiments mayreceive a first input for photographing the two eyes of the user.

The device 100 according to an embodiment may receive the first inputused to photograph the two eyes of the user. For example, the device 100may receive the first input requesting the device 100 to photograph thetwo eyes of the user. For example, the device 100 may photograph the twoeyes of the user upon receiving a touch input of the user. As anotherexample, the device 100 may receive the first input of requesting thedevice 100 to photograph the two eyes of the user when a pre-setapplication is performed, without any particular input from the user.The device 100 may receive the first input from an external source ofthe device 100 or obtain the first input from some components includedin the device 100.

In operation S220, when the first input is received by the device 100,the first binocular distance between the two eyes of the user in thefocal plane of the camera may be determined by the device 100, accordingto an embodiment, through the image sensor of the camera of the device100.

The focal plane of the camera may denote a plane where incident light isgathered by the lens of the camera. For example, the focal plane may bea virtual plane where light incident through the lens is gathered whenthe lens of the camera is set to infinity.

The image sensor may be located in the focal plane of the camera. Theimage sensor may obtain an image by sensing the image formed on theimage sensor.

The device 100 may determine, through photographing according to thefirst input, the first binocular distance that is the distance betweenthe two eyes of the user in the focal plane of the camera. The firstbinocular distance may be the distance between the two eyes of the userin the focal plane of the camera at a point of time according to thefirst input. When the image sensor is located in the focal plane of thecamera, a distance between an image of a first eyeball of the user andan image of a second eyeball of the user, which are formed on the imagesensor through the photographing according to the first input, may bethe first binocular distance. When the image sensor is located in thefocal plane of the camera, the device 100 may determine the firstbinocular distance through the image sensor. The focal length of thecamera may denote a distance from the lens of the camera to the focalplane of the camera or to the image sensor.

The camera may be located inside or outside the device 100. When thecamera is located outside the device 100, the device 100 may obtaininformation sensed by the image sensor from the camera, and determinethe first binocular distance by using the sensed information.Alternatively, when the camera is included in the device 100, the device100 may determine the first binocular distance by using the camerainside the device 100.

In operation S230, the device 100 according to an embodiment may receivea second input for photographing the two eyes of the user.

The device 100 may receive the second input used to photograph the twoeyes of the user after the first input is received. For example, thedevice 100 may sequentially receive the first and second inputsrequesting the device 100 to photograph the two eyes of the user. Forexample, the device 100 may photograph the two eyes of the useraccording to a touch input of the user received after the first input isreceived. As another example, the device 100 may sequentially receivethe first and second inputs when the pre-set application is performed,without a particular input from the user. The device 100 may receive thesecond input from an external source of the device 100 or obtain thesecond input from some components included in the device 100.

In operation S240, upon receiving the second input, the device 100according to an embodiment may determine, through the image sensor, thesecond binocular distance between the two eyes of the user in the focalplane.

The device 100 may determine the second binocular distance that is thedistance between the two eyes of the user formed on the focal plane ofthe camera, through photographing according to the second input. Thesecond binocular distance may be the distance between the two eyes ofthe user in the focal plane of the camera at a point of time accordingto the second input. When the image sensor is located in the focal planeof the camera, a distance between an image of the first eyeball of theuser and an image of the second eyeball of the user, the images beingformed on the image sensor through the photographing according to thesecond input, may be the second binocular distance. When the imagesensor is located in the focal plane of the camera, the device 100 maydetermine the second binocular distance through the image sensor.

The camera may be located inside or outside the device 100. When thecamera is located outside the device 100, the device 100 may obtaininformation sensed by the image sensor from the camera, and determinethe second binocular distance by using the sensed information.Alternatively, when the camera is included in the device 100, the device100 may determine the second binocular distance by using the camerainside the device 100.

The first binocular distance and the second binocular distance may bedifferent from each other. For example, the first and second binoculardistances may be different from each other when distances from the lensof the camera to the two eyes of the user at a photographing timeaccording to reception of the first input and at a photographing timeaccording to reception of the second input are different from eachother.

In operation S250, the device 100 according to an embodiment may obtaina variation of the distance between the device 100 and the user fromwhen the first input is received to when the second input is received.The device 100 may detect motion of the device 100 from when the firstinput is received to when the second input is received, and obtain thevariation of the distance based on the detected motion of the device100. A point of time when the first input or the second input isreceived may include not only a physical point of time when the firstinput or the second input is received, but also a point of time when thedevice 100 performs a pre-set operation according to the first input orthe second input. For example, the device 100 may obtain the variationof the distance from the lens of the camera to the two eyes of the userfrom the photographing time according to the reception of the firstinput to the photographing time according to the reception of the secondinput.

The device 100 may obtain the variation of the distance by usinginformation obtained from the sensor. The sensor located inside oroutside the device 100 may obtain sensing information. For example, thesensor may obtain motion information. When the sensor is located insidethe device 100 or included in an apparatus like the device 100, motiondetected by the sensor may indicate the motion of the device 100. Inthis case, the sensor may obtain motion information of the device 100.The sensor may obtain the motion information of the device 100, and thedevice 100 may obtain the motion information of the device 100 from thesensor.

The device 100 may determine a moving distance and a moving direction ofthe device 100 from the photographing time according to the reception ofthe first input to the photographing time according to the reception ofthe second input, by using the motion information obtained from thesensor. The device 100 may determine the variation of the distance fromthe lens of the camera to the two eyes of the user from thephotographing time according to the reception of the first input to thephotographing time according to the reception of the second input,according to the determined moving distance and moving direction.

The device 100 according to an embodiment may obtain the focal length ofthe camera. The focal length of the camera may denote a distance fromthe lens of the camera to the focal plane of the camera. Also, when theimage sensor is located in the focal plane, the focal length of thecamera or the focal length of the lens of the camera may denote thedistance from the lens of the camera to the image sensor. The focallength of the camera may be determined according to hardwarecharacteristics of the camera. For example, the focal length of thecamera may be determined according to a refractive index of the lens ofthe camera. Information indicating the focal length of the camera may bestored in the device 100 or received from an external source of thedevice 100. The device 100 may determine the focal length of the cameraaccording to the obtained information indicating the focal length of thecamera.

In operation S260, the device 100 according to an embodiment maydetermine the distance between the device 100 and the user based on thefirst binocular distance determined in operation S220, the secondbinocular distance determined in operation S240, and the variation ofthe distance obtained in operation S250.

The device 100 may determine the distance from the lens of the camera tothe two eyes of the user at the photographing time according to thereception of the first input and/or the distance from the lens of thecamera to the two eyes of the user at the photographing time accordingto the reception of the second input, by comparing the first binoculardistance and the second binocular distance. For example, the device 100may determine the distance from the lens of the camera to the two eyesof the user at the photographing time according to the reception of thefirst input and/or the distance from the lens of the camera to the twoeyes of the user at the photographing time according to the reception ofthe second input, by using the first binocular distance determined inoperation S220, the second binocular distance determined in operationS240, and the variation of the distance and the focal length of thecamera obtained in operation S250. An example of determining thedistance from the lens of the camera to the two eyes of the user will bedescribed below with reference to FIG. 3.

The distance between the two eyes of the user in the focal plane maydenote the distance between images of the two eyes of the user in thefocal plane.

The distance from the lens of the camera to the two eyes of the user maybe defined according to a pre-set method. For example, the distance fromthe lens of the camera to the two eyes of the user may be a distancebetween the lens of the camera and a virtual segment connecting the twoeyes. As another example, the distance from the lens of the camera tothe two eyes of the user may be a shortest distance between a firstplane parallel to the lens of the camera and located on the lens of thecamera and a second plane parallel to the lens of the camera andincluding the two eyes of the user. However, examples of the distancefrom the lens of the camera to the two eyes of the user are not limitedto the above.

FIG. 3 is a diagram for describing an example in which a devicedetermines the distance from the device to a user by using a firstbinocular distance and a second binocular distance, according to anembodiment of the present disclosure.

Referring to FIG. 3, the device 100 may determine a first binoculardistance W₁ and a second binocular distance W₂ respectively in a firstcase 310 when two eyes 311 of the user are photographed at a first pointof time and a second case 320 when the two eyes 311 of the user arephotographed at a second point of time.

The device 100 according to an embodiment may obtain the first binoculardistance W₁ at the first point of time. An image 312 of the two eyes 311of the user at the first point of time may be formed in a focal plane ofa lens 300. An image sensor located in the focal plane may sense theimage 312. The device 100 may determine the first binocular distance W₁in the image 312 obtained through the image sensor of a camera at thefirst point of time.

The device 100 according to an embodiment may obtain the secondbinocular distance W₂ at the second point of time. An image 322 of thetwo eyes 311 of the user at the second point of time may be formed inthe focal plane of the lens 300. The image sensor located in the focalplane may sense the image 322. The device 100 may determine the secondbinocular distance W₂ from the image 322 obtained through the imagesensor of the camera at the second point of time.

The device 100 according to an embodiment may obtain information about afocal length D_(F) that is a distance from the lens 300 to the focalplane. The focal length D_(F) may be determined according to hardwarecharacteristics of the camera or lens. Also, when the image sensor islocated in the focal plane, a distance between the lens 300 and theimage sensor may be the focal length D_(F).

The device 100 according to an embodiment may obtain a variation ΔD of adistance that is a variation of the distance from the two eyes 311 ofthe user to the lens 300 between the first point of time and the secondpoint of time. The variation ΔD may indicate a difference between afirst distance D₁ that is the distance from the two eyes 311 to the lens300 at the first point of time and a second distance D₂ that is thedistance from the two eyes 311 to the lens 300 at the second point oftime. The device 100 may determine the variation ΔD by using a sensorincluded in the device 100 or by using data received from an externalsource of the device 100. Alternatively, the device 100 may receiveinformation indicating the variation ΔD from an external source of thedevice 100.

A distance W_(E) between the two eyes 311 of the user may be the same atthe first point of time and the second point of time. Accordingly, thedevice 100 may determine the first distance D₁ that is the distance fromthe two eyes 311 to the lens 300 at the first point of time and thesecond distance D₂ that is the distance from the two eyes 311 to thelens 300 at the second point of time, by using the first binoculardistance W₁, the second binocular distance W₂, the focal length D_(F),and the variation ΔD.

For example, the device 100 may determine the first distance D₁ and/orthe second distance D₂ by using Equations 1 through 3.

W_(E):D₁=W₁:D_(F)  Equation 1

W_(E):D₂=W₂:D_(F)  Equation 2

ΔD=D ₂ −D ₁  Equation 3

The device 100 may obtain the first distance D₁, the second distance D₂,and the distance W_(E) by using the obtained or determined firstbinocular distance W₁, second binocular distance W₂, focal length D_(F),and variation ΔD, and Equations 1 through 3.

Also, the device 100 according to an embodiment may obtain a pluralityof distances from the lens 300 to the two eyes 311, and output a valueindicating a distance from the lens 300 to the two eyes 311, which ismost recently obtained. For example, the device 100 may output thesecond distance D₂ that is a value more recently obtained from among thefirst distance D₁ and the second distance D₂. In this case, the device100 may display the value indicating the second distance D₂.

FIG. 4 is a flowchart of a method by which a device determines adistance from the device to a user by using sensing information,according to an embodiment of the present disclosure.

Referring to FIG. 4, the device 100 according to an embodiment mayobtain the sensing information in operation S410.

The sensing information may be information obtained from a sensorlocated inside or outside the device 100.

The sensor according to an embodiment may include a motion sensor. Themotion sensor may detect motion of the device 100. The device 100 maydetect the motion of the device 100 by using the motion sensor. Forexample, when the motion sensor is included in or attached to the device100, the device 100 may determine motion detected by the motion sensoras the motion of the device 100. The device 100 may obtain motioninformation obtained by the motion sensor. For example, motioninformation generated by the motion sensor located outside the device100 may be received from the motion sensor by the device 100.

The sensor according to an embodiment may include a proximity sensor.The proximity sensor may obtain information indicating a variation of adistance from the proximity sensor to the device 100. Also, the device100 may receive information sensed by the proximity sensor from theproximity sensor located inside or outside the device 100.

Examples of a sensor used by the device 100 to determine a distance mayinclude various sensors used to determine a variation of a distancebetween the device 100 and the user, and are not limited thereto. Forexample, the sensor used by the device 100 to determine a distance mayinclude a motion sensor, an acceleration sensor, a gyroscope sensor, anda proximity sensor.

In operation S420, the device 100 according to an embodiment maydetermine a variation of a distance from a lens of a camera to two eyesof the user based on the sensing information obtained in operation S410.

The device 100 according to an embodiment may obtain the variation ofthe distance from the lens of the camera to the two eyes of the user,between a first point of time and a second point of time. The variationmay indicate a difference between a first distance, i.e., a distancefrom the two eyes of the user to the lens of the camera at the firstpoint of time, and a second distance, i.e., a distance from the two eyesof the user to the lens of the camera at the second point of time.

When the user is sequentially photographed so as to determine adistance, a point of time corresponding to first photographing may bethe first point of time, and a point of time corresponding to secondphotographing may be the second point of time. For example, when the twoeyes of the user are sequentially photographed two times, the firstpoint of time may be a point of time when a first input is received forfirst photographing, and the second point of time may be a point of timewhen a second input is received for second photographing. As anotherexample, when the two eyes of the user are sequentially photographed twotimes, the first point of time may be a point of time when firstphotographing is performed and the second point of time may be a pointof time when second photographing is performed.

In operation S430, the device 100 according to an embodiment maydetermine a distance from the lens of the camera to the two eyes of theuser based on a first binocular distance, a second binocular distance, avariation of the distance, and a focal length of the camera.

According to an embodiment, the first binocular distance may be adistance between the two eyes of the user in a focal plane of the cameraat the first point of time, the second binocular distance may be adistance between the two eyes of the user in the focal plane of thecamera at the second point of time, the variation of the distance may bethe difference between the first distance, i.e., the distance from thetwo eyes of the user to the lens of the camera at the first point oftime, and the second distance, i.e., the distance from the two eyes ofthe user to the lens of the camera at the second point of time, and thefocal length of the camera may be a distance from the lens of the camerato the focal plane of the camera. A distance from the lens of the camerato the two eyes of the user may include the first distance and/or thesecond distance. For example, the distance from the lens of the camerato the two eyes of the user may be updated to a recent value inreal-time.

An example of determining the distance from the lens of the camera tothe two eyes of the user have been described above with reference toFIG. 3.

FIG. 5 is a diagram for describing an example in which a devicedetermines a distance from the device to a user by using a firstbinocular distance, a second binocular distance, and a sensing unit,according to an embodiment of the present disclosure.

An example of determining a distance from the device 100 to the user inFIG. 5 has been described above with reference to FIG. 3.

Referring to FIG. 5, a sensing unit may be used to obtain the variationΔD of the distance. For example, the sensing unit may generate sensinginformation used to determine the variation ΔD of the distance, i.e., avariation of a distance from the two eyes 311 of the user to the lens300 between a first point of time and a second point of time.

The sensing unit may include various sensors used to determine thevariation of the distance between the device 100 and the user. Forexample, the sensing unit may include a motion sensor, an accelerationsensor, a gyroscope sensor, and a proximity sensor.

FIG. 6 is a diagram for describing an example in which a devicedetermines a distance from the device to a user by using an eyeball sizeof the user, according to an embodiment of the present disclosure. Forexample, the device may determine the distance from the lens of thecamera to an eyeball of the user by photographing one eyeball of theuser by using one camera.

Referring to FIG. 6, the device 100 according to an embodiment mayobtain user information and predict the eyeball size of the user byusing the obtained user information. The device 100 may determine thesize of the eyeball of the user in the focal plane of the camera. Thedevice 100 may determine the distance from the lens of the camera to theeyeball of the user by comparing the predicted eyeball size and thedetermined eyeball size.

The device 100 according to an embodiment may determine the distancefrom the lens of the camera to the eyeball of the user through onephotographing operation using one camera.

Alternatively, the device 100 may update the distance from the lens ofthe camera to the eyeball of the user through a plurality ofphotographing operations using one camera.

FIG. 7 is a flowchart of a method by which a device determines adistance from the device to a user by using an eyeball size of the user,according to an embodiment of the present disclosure.

Referring to FIG. 7, the device 100 according to an embodiment maypredict the eyeball size of the user based on user information stored inthe device 100 in operation S710.

The device 100 according to an embodiment may store the userinformation. For example, the device 100 may store the user informationabout a race, a gender, a nationality, an age, etc., of the user.

The device 100 may predict the eyeball size of the user according tostatistics data based on the user information stored in the device 100.For example, the device 100 may predict the eyeball size of the useraccording to a pre-set table indicating eyeball sizes according toraces, genders, ages, etc.

Alternatively, the device 100 may store information about the eyeballsize of the user. When the device 100 stores the information about theeyeball size of the user, the device 100 may determine the eyeball sizeof the user without the predicting.

The user information may be stored in the device 100 or may be receivedfrom an external source of the device 100. For example, the device 100may receive and use the user information from a server (not shown) via awireless communication.

In operation S720, the device 100 according to an embodiment maydetermine the size of the eyeball of the user in the focal plane of thecamera.

The device 100 may determine the size of the eyeball of the user in thefocal plane of the camera through photographing. The size of the eyeballin the focal plane of the camera may be the size of the eyeball of theuser in the focal plane of the camera. When the image sensor is locatedin the focal plane of the camera, the size of the eyeball of the userformed as an image on the image sensor through photographing may be thesize of the eyeball of the user in the focal plane of the camera. Whenthe image sensor is located in the focal plane of the camera, the device100 may determine the size of the eyeball in the focal plane of thecamera through the image sensor.

The size of the eyeball in the focal plane of the camera may varyaccording to the distance from the eyeball to the lens of the camera.For example, the size of the eyeball in the focal plane of the cameramay decrease when the distance from the lens of the camera to theeyeball increases.

The camera may be located inside or outside the device 100. When thecamera is outside the device 100, the device 100 may obtain informationsensed by the image sensor from the camera, and determine the size ofthe eyeball in the focal plane of the camera by using the sensedinformation. Alternatively, when the camera is included in the device100, the device 100 may determine the size of the eyeball in the focalplane of the camera by using the camera inside the device 100.

In operation S730, the device 100 according to an embodiment maydetermine the distance from the lens of the camera to the eyeball basedon the eyeball size predicted in operation S710 and the eyeball sizedetermined in operation S720. Also, the device 100 may use the focallength of the camera while determining the distance from the lens of thecamera to the eyeball.

An example of determining the distance from the lens of the camera tothe two eyes of the user by photographing the eyeball will be describedbelow with reference to FIG. 8.

FIG. 8 is a diagram for describing an example in which a devicedetermines a distance from the device to a user by using a predictedeyeball size, according to an embodiment of the present disclosure.

Referring to FIG. 8, the device 100 may determine a first eyeball sizeW′₁ and a second eyeball size W′₂ respectively in a first case 810 whenan eyeball 811 of the user is photographed at a first point of time andin a case 820 when the eyeball 811 of the user is photographed at asecond point of time.

The device 100 according to an embodiment may obtain the first eyeballsize W′₁ at the first point of time. An image 812 of the eyeball 811 ofthe user at the first point of time may be formed in a focal plane ofthe lens 300. An image sensor located in the focal plane may sense theimage 812. The device 100 may determine the first eyeball size W′₁ inthe image 812 obtained through the image sensor of the camera at thefirst point of time.

The device 100 according to an embodiment may obtain the second eyeballsize W′₂ at the second point of time. An image 822 of the eyeball 811 ofthe user at the second point of time may be formed in the focal plane ofthe lens 300. The image sensor located in the focal plane may sense theimage 822. The device 100 may determine the second eyeball size W′₂ inthe image 822 obtained through the image sensor of the camera at thesecond point of time.

The device 100 according to an embodiment may obtain information about afocal length D′_(F) that is a distance from the lens 300 to the focalplane. An example of the focal length D′_(F) may correspond to thatdescribed above with reference to FIG. 3.

The eyeball size W′_(E) may be the same at the first point of time andthe second point of time. Also, the eyeball size W′_(E) of the user maybe predicted or determined based on user information.

The device 100 according to an embodiment may not pre-determine avariation ΔD′ of a distance that is a variation of a distance from theeyeball 811 to the lens 300 between the first point of time and thesecond point of time, but may determine a first distance D′₁ that is adistance from the eyeball 811 to the lens 300 at the first point of timeor a second distance D′₂ that is a distance from the eyeball 811 to thelens 300 at the second point of time. For example, the device 100 maydetermine the first distance D′₁ or the second distance D′₂ withoutsensing information received from a motion sensor.

For example, the device 100 may determine the first distance D′₁ fromthe eyeball 811 to the lens 300 at the first point of time by using thefirst eyeball size W′₁, the eyeball size W′_(E), and the focal lengthD′_(F). At this time, Equation 4 may be used.

W′_(E):D′₁=W′₁:D′_(F)  Equation 4

As another example, the device 100 may determine the second distance D′₂from the eyeball 811 to the lens 300 at the second point of time byusing the second eyeball size W′₂, the eyeball size W′_(E), and thefocal length D′_(F). At this time, Equation 5 may be used.

W′_(E):D′₂=W′₂:D′_(F)  Equation 5

As another example, the device 100 may determine the second distance D′₂by using the first eyeball size W′₁, the second eyeball size W′₂, andthe first distance D′₁. At this time, Equation 6 may be used.

D′ ₂ =D′ ₁*(W′ ₁ /W′ ₂)  Equation 6

As described above, the device 100 may determine the first distance D′₁or the second distance D′₂ by using Equations 4 through 6.

The device 100 according to an embodiment may determine the distancefrom the lens 300 of the camera to the eyeball 811 to be a pre-set valuewhen a size of an eyeball in the focal plane is a pre-set size. Thedevice 100 may determine the distance from the lens 300 to the eyeball811 without information about the focal length D′_(F). For example, thedevice 100 may determine the first distance D′₁ by using the firsteyeball size W′₁ and the eyeball size W′_(E). The device 100 maydetermine the first distance D′₁ to be a pre-set value (for example, 20cm) when the first eyeball size W′₁ is a pre-set size determinedaccording to the eyeball size W′_(E).

Also, the device 100 according to an embodiment may obtain a pluralityof distances from the lens 300 to the eyeball 811, and output a valueindicating a most recently obtained distance from the lens 300 to theeyeball 811. For example, the device 100 may output the second distanceD′₂ that is a more recently obtained value from among the first distanceD′₁ and the second distance D′₂. In this case, the device 100 maydisplay a value indicating the second distance D′₂.

FIG. 9 is a diagram for describing an example in which a device uses aguideline to determine a distance, according to an embodiment of thepresent disclosure.

Referring to FIG. 9, the device 100 according to an embodiment matches aguideline to one eyeball from among eyes of a user. For example, thedevice 100 may match a first guideline 910 to a right eyeball of the twoeyes of the user.

The device 100 according to an embodiment may match the guideline to theeyeball of the user via a pre-set method, and determine a size of theeyeball based on the matched guideline. For example, the device 100 maymatch the first guideline 910 having a pre-set size to the right eyeballof the user, and determine the pre-set size of the first guideline 910as the size of the eyeball.

The device 100 according to an embodiment may match the guideline to theeyeball of the user via a pre-set method, and determine a distance froma lens of a camera to the eyeball of the user based on the matchedguideline. The device 100 may match the guideline to the eyeball of theuser and, when the size of the eyeball and the size of the guideline arematched in a pre-set method, determine the distance between the lens ofthe camera and the eyeball of the user to be a pre-set value. Forexample, the device 100 may determine the distance between the lens ofthe camera and the eyeball of the user to be 20 cm at the moment when aneyelid of the user is included in the first guideline 910 having thepre-set size.

A case in which the device 100 according to an embodiment matches aguideline to each of the two eyes of the user will now be described. Forexample, the device 100 may match the first guideline 910 to the righteyeball and a second guideline 920 to a left eyeball from among the twoeyes of the user.

The device 100 according to an embodiment may match guidelinesrespectively to the two eyes of the user via a pre-set method, anddetermine a distance between the two eyes based on a distance betweenthe guidelines. For example, the device 100 may match the firstguideline 910 to the right eyeball of the user, match the secondguideline 920 to the left eyeball of the user, and determine thedistance between the two eyes based on a distance between the first andsecond guidelines 910 and 920.

FIG. 10 is a flowchart of a method by which a device determines adistance from the device to a user by using a degree of tilt of thedevice, according to an embodiment of the present disclosure.

Referring to FIG. 10, the device 100 according to an embodiment maydetermine the degree of tilt of the device 100 in operation S1010.

The device 100 according to an embodiment may determine the degree oftilt of the device 100 based on a pre-set reference direction. Forexample, the device 100 may determine the degree of tilt of the device100 based on a gravity direction. For example, the device 100 maydetermine degree of tilt of the device 100 according to an angle formedby the gravity direction and a direction of a line perpendicular to asurface of a lens of a camera. The device 100 may determine the degreeof tilt of the device 100 to be 0° when the angle formed by the gravitydirection and the direction of the line perpendicular to the surface ofthe lens of the camera is 90°.

The device 100 according to an embodiment may determine the degree oftilt of the device 100 by using a sensor. For example, the device 100may determine the degree of tilt of the device 100 by using a gravitysensor, a tilt sensor, an acceleration sensor, or a gyroscope sensor,but a type of the sensor is not limited thereto.

In operation S1020, the device 100 according to an embodiment maydetermine a line of sight of the user. For example, the device 100 maydetermine the line of sight of the user by using an image of aphotographed eyeball.

In operation S1030, the device 100 according to an embodiment may updatea distance from the lens of the camera to two eyes of the user based ona value determined according to the degree of tilt of the device 100determined in operation S1010 and the line of sight of the userdetermined in operation S1020.

For example, the device 100 may update the distance from the lens of thecamera to the two eyes of the user, which is determined according to themethod described above with reference to FIGS. 2 through 9, by using anangle determined according to the degree of tilt of the device 100determined in operation S1010 and the line of sight of the userdetermined in operation S1020.

FIG. 11 is a diagram for describing an example in which a devicedetermines a distance from the device to a user when a camera is locatedhigher than a height of two eyes of the user according to an embodimentof the present disclosure.

Referring to FIG. 11, a distance DUu from the device 100 to the user maybe determined according to a distance Du from a camera 1610 to two eyesof the user and an angle θ. For example, when the camera 1610 is locatedhigher than the height of two eyes of the user, the device 100 maydetermine the distance DUu from the device 100 to the user according toEquation 7. As illustrated in FIG. 11, the angle θ may denote an anglebetween a gravity direction and a direction of tilt of the device 100.

DUu=Du/cos θ  Equation 7

FIG. 12 is a diagram for describing an example in which a devicedetermines a distance from the device to a user, according to anembodiment, when a camera is located lower than a height of two eyes ofthe user.

Referring to FIG. 12, a distance DLu from the device 100 to the user maybe determined according to the distance Du from the camera 1610 to twoeyes of the user and the angle θ. For example, when the camera 1610 islocated lower than the height of two eyes of the user, the device 100may determine the distance DLu from the device 100 to the user accordingto Equation 8. Referring to FIG. 12, the angle θ may denote an anglebetween a gravity direction and a direction of tilt of the device 100.

DLu=Du*cos θ  Equation 8

FIG. 13 is a flowchart of a method by which a device processes an imagebased on a distance from the device to a user and user information,according to an embodiment of the present disclosure.

Referring to FIG. 13, the device 100 or a server, according to anembodiment, may determine a distance from a lens of a camera to two eyesof the user in operation S1310. A method by which the device 100 or theserver determines the distance from the lens of the camera to the twoeyes of the user has been described above with reference to FIGS. 1through 12.

In operation S1320, the device 100 or the server, according to anembodiment, may obtain the user information. For example, the device 100or the server may store the user information. As another example, thedevice 100 or the server may receive the user information from anexternal source (for example, an external device) of the device 100 orthe server.

The user information may include visual acuity information of the user.For example, the user information may include information aboutcharacteristics (for example, nearsightedness, farsightedness, orastigmatism) of an eyeball of the user and about visual acuity (forexample, 1.0 or 0.5) of the user.

The device 100 or the server, according to an embodiment, may provide aservice that requires the user to log in. The device 100 or the servermay obtain the user information during a login process. For example, thedevice 100 or the server may obtain visual acuity information input bythe user to subscribe to the service that requires the user to log in,during the login process. As another example, the device 100 or theserver may obtain information about an enlargement magnification set bythe service that requires the user to log in, during the login process.

In operation S1330, the device 100 or the server, according to anembodiment, may process an image to be displayed according to thedistance from the lens of the camera to the two eyes of the userdetermined in operation S1310, and the user information obtained inoperation S1320. The image processed by the device 100 may be a2-dimensional (2D) image and/or a 3D image.

The device 100 according to an embodiment may change a location of afocal plane of the image based on the visual acuity information of theuser stored in the device 100. For example, when a characteristic of theeyeball of the user is nearsightedness, the device 100 may change thelocation of the focal plane of the image being displayed from a locationof the device 100 to a location closer to the two eyes of the user. Asanother example, when the characteristic of the user is farsightedness,the device 100 may change the location of the focal plane of the imagebeing displayed from the location of the device 100 to a locationfarther from the two eyes of the user.

The device 100 according to an embodiment may adjust directivity oflight being displayed based on the visual acuity information stored inthe device 100.

When the image to be displayed is processed, the device 100 according toan embodiment may use a light field display method or a pin hole maskmethod.

The server according to an embodiment may determine the location of thefocal plane of the image based on the visual acuity information storedin the server, and obtain a light field image according to thedetermined location of the focal plane. For example, when thecharacteristic of the eyeball of the user is nearsightedness, the servermay generate the light field image, in which the location of the focalplane of the image to be displayed is determined to be changed from thelocation of the device 100 to the location closer to the two eyes of theuser, and transmit the light field image to the device 100. As anotherexample, when the characteristic of the eyeball of the user isfarsightedness, the server may generate the light field image, in whichthe location of the focal plane of the image to be displayed isdetermined to be changed from the location of the device 100 to thelocation farther from the two eyes of the user, and transmit the lightfield image to the device 100.

The server according to an embodiment may obtain the light field imagebased on setting information of the user stored in the server. Forexample, the server may generate the light field image enlarged orreduced according to information about an enlargement magnification setby the user, and transmit the light field image to the device 100.

The device 100 according to an embodiment may display the light fieldimage received from the server. Also, the displayed light field imagemay be displayed after being modified according to a user environment.For example, the light field image may be displayed with brightnessdetermined according to ambient light of the device 100.

When the server performs operations on the light field image, throughputrequired by the device 100 may be reduced.

FIG. 14 is a diagram for describing an example in which a devicedetermines a location of a focal plane of an image based on a distancefrom the device to a user and user information, according to anembodiment of the present disclosure.

Referring to FIG. 14, the device 100 according to an embodiment maydetermine characteristics of an eyeball of the user and visual acuity ofthe user based on the user information.

When the user is farsighted, an image of an image displayed on a display1401 may be formed behind a retina 1404 according to the visual acuityof the user (for example, in a first focal plane 1405 of an eye lens1403, a second focal plane 1406 of the eye lens 1403, a third focalplane 1407 of the eye lens 1403, or a fourth focal plane 1408 of the eyelens 1403). When a distance from a focal plane of the eye lens 1403, inwhich the image is formed, to the eye lens 1403 is longer than adistance from the retina 1404 to the eye lens 1403, the device 100 mayprocess the image such that a focal plane of the image being displayedis located behind the display 1401.

When the user is nearsighted, the image of the image displayed on thedisplay 1401 may be formed in front of the retina 1404 according to thevisual acuity of the user. When the distance from the focal plane of theeye lens 1403, in which the image is formed, to the eye lens 1403 isshorter than the distance from the retina 1404 to the eye lens 1403, thedevice 100 may process the image such that the focal plane of the imagebeing displayed is located in front of the display 1401. For example,the device 100 may process the image such that the focal plane of theimage is located on a zero^(th) focal plane 1402.

FIG. 15 is a diagram for describing an example in which a deviceprovides an image to a user by using a pinhole mask, according to anembodiment of the present disclosure.

Referring to FIG. 15, a distance between the display 1401 and a pin holemask 1501 may be pre-determined. Also, the device 100 may process animage such that a focal plane 1502 of the processed image is located atthe pin hole mask 1501. The device 100 may control directivity of lightof the image being displayed such that an image of the image beingdisplayed on the display 1401 is formed on the retina 1404.

FIG. 16 is a flowchart of a method by which a device processes an imageaccording to a type of content being displayed, according to anembodiment.

Referring to FIG. 16, the device 100 according to an embodiment maydetermine a distance from a lens of a camera to two eyes of a user inoperation S1610. An example by which the device 100 determines thedistance from the lens of the camera to the two eyes of the user hasbeen described above with reference to FIGS. 1 through 12.

In operation S1620, the device 100 according to an embodiment maydetermine the type of the content being displayed. For example, thedevice 100 may determine whether the content being displayed includestext. As another example, the device 100 may determine what percentageof the content being displayed is text.

In operation S1630, the device 100 according to an embodiment mayprocess an image to be displayed according to the distance determined inoperation S1610 and the type of the content determined in operationS1620. For example, when the percentage of the text in the content beingdisplayed is equal to or higher than a certain value, the device 100 mayperform outline emphasis filtering on the image to be displayed.

FIG. 17 is a flowchart of a method by which a device processes an imagebased on whether a type of content being displayed is video or text,according to an embodiment of the present disclosure.

Referring to FIG. 17, the device 100 according to an embodiment mayanalyze a display source type in operation S1710. For example, thedevice 100 may distinguish a plurality of pieces of content included inan image being displayed.

In operation S1720, the device 100 according to an embodiment maydetermine a type of the image being displayed, according to a result ofthe analyzing in operation S1710. For example, the device 100 maydetermine whether the content included in the image being displayed isvideo, text, or both video and text.

In operation S1731, the device 100 according to an embodiment maygenerate a display data block.

In operation S1732, the device 100 according to an embodiment maydetermine whether the display data block generated in operation S1731 isa video block or a text block.

In operation S1733, the device 100 according to an embodiment mayaggregate display data.

In operation S1741, the device 100 according to an embodiment maygenerate a light field image.

In operation S1742, the device 100 according to an embodiment mayperform adaptive high-pass filtering. At this time, sharpness of anoutline of text to be displayed may increase.

In operation S1743, the device 100 according to an embodiment may updatethe display data.

In operation S1751, the device 100 according to an embodiment mayextract a motion vector of the image.

In operation S1752, the device 100 according to an embodiment maygenerate an image mapping table.

In operation S1753, the device 100 according to an embodiment mayperform light field image copying on a mapping region.

In operation S1754, the device 100 according to an embodiment may updatethe display data.

FIG. 18 is a diagram for describing an example in which a devicedetermines a distance between the device and a user, and displays thedetermined distance, according to an embodiment of the presentdisclosure.

Referring to FIG. 18, the device 100 according to an embodiment maydetermine and display the distance between the device 100 and the userin real-time. For example, the device 100 may continuously update adistance between a lens of a camera and two eyes of the user whilefilming video. Also, the updated distance between the lens of the cameraand the two eyes of the user may be continuously displayed in a pre-setregion 1810 of a display. The distance displayed in the pre-set region1810 may be a most recently determined distance between the user and thedevice 100.

The device 100 according to an embodiment may display the two eyes ofthe user detected on a screen. For example, the device 100 may displaylocations of the two eyes of the user detected on a currently capturedimage by using dots and lines. In this case, the user may intuitivelyrecognize that the displayed distance is the distance from the device100 to the two eyes.

FIG. 19 is a diagram for describing an example in which a deviceprovides a guiding instruction to a user in order to photograph two eyesof the user, according to an embodiment of the present disclosure.

Referring to FIG. 19, when the device 100 according to an embodimentdetects the two eyes of the user from a first region 1910, the device100 may display a guideline in the first region 1910. Also, the device100 may display, in a second region 1920, an instruction, such as asentence, requesting the user to adjust a location of the camera 1610such that the eyes are located in the guideline. When the two eyes arelocated in the first region 1910, the device 100 may determine adistance from the device 100 to the two eyes of the user by recognizingthe two eyes of the user. The distance from the device 100 to the twoeyes of the user may include a distance from a lens of the camera 1610to the two eyes of the user. Alternatively, even when only one eyeballfrom among the two eyes of the user is detected in the first region1910, the device 100 may determine a distance from the eyeball to thedevice 100 by using a size of the detected eyeball. The distance fromthe eyeball to the device 100 may include a distance from the lens ofthe camera 1610 to the eyeball.

FIGS. 20 and 21 are block diagrams of a device according to variousembodiments of the present disclosure.

Referring to FIG. 20, the device 100 according to an embodiment mayinclude the camera 1610, a processor 1300, and the sensing unit 1400.However, not all components shown in FIG. 20 are essential components ofthe device 100. The device 100 may include more or fewer components thanthose shown in FIG. 20.

Referring to FIG. 21, the device 100 according to an embodiment mayfurther include, in addition to the camera 1610, the processor 1300, andthe sensing unit 1400, a user input unit 1100, an output unit 1200, acommunication unit 1500, an audio/video (A/V) input unit 1600, and amemory 1700.

The user input unit 1100 is a unit for a user to input data forcontrolling the device 100. Examples of the user input unit 1100 includea keypad, a dome switch, a touch pad (a touch capacitance type, apressure resistance film type, an infrared light detecting type, asurface ultrasound conducting type, an integral tension measuring type,or a piezo-effect type), a jog wheel, and a jog switch, but are notlimited thereto.

The user input unit 1100 may capture an image (for example, photographtwo eyes of the user), or receive a user input for determining adistance from the device 100 to the user.

The output unit 1200 may output an audio signal, a video signal, or avibration signal, and may include a display unit 1210, a sound outputunit 1220, and a vibration motor 1230.

The display unit 1210 displays information processed by the device 100.For example, the display unit 1210 may display an image captured throughthe camera 1610. As another example, the display unit 1210 may displayan image processed based on the determined distance between the device100 and the user.

Meanwhile, when the display unit 1210 is configured as a touch screen byforming a layer structure with a touch pad, the display unit 1210 mayalso be used as an input device as well as an output device. The displayunit 1210 may include at least one of a liquid crystal display (LCD), athin-film-transistor liquid-crystal display (TFT-LCD), an organiclight-emitting diode (OLED), a flexible display, a 3D display, and anelectrophoretic display. Also, according to an embodiment of the device100, the device 100 may include at least two display units 1210. Here,the at least two display units 1210 may be disposed to face each otherby using a hinge.

The sound output unit 1220 outputs audio data received from thecommunication unit 1500 or stored in the memory 1700. Also, the soundoutput unit 1220 outputs a sound signal related to a function performedby the device 100, such as a call signal reception sound, a messagereception sound, or an alarm sound. The sound output unit 1220 mayinclude a speaker or a buzzer.

The vibration motor 1230 may output a vibration signal. For example, thevibration motor 1230 may output a vibration signal corresponding to anoutput of audio data or video data, for example, a call signal receptionsound or a message reception sound. Also, the vibration motor 1230 mayoutput a vibration signal when a touch screen is touched.

The processor 1300 controls overall operations of the device 100. Forexample, the processor 1300 may generally control the user input unit1100, the output unit 1200, the sensing unit 1400, the communicationunit 1500, and the A/V input unit 1600 by executing programs stored inthe memory 1700.

The processor 1300 may receive a first input for photographing the twoeyes of the user.

The processor 1300 according to an embodiment may receive the firstinput used to photograph the two eyes of the user. The processor 1300may photograph the two eyes of the user when a touch input of the useris received. Alternatively, the processor 1300 may receive the firstinput requesting the device 100 to photograph the two eyes of the userwhen a pre-set application is performed, without a particular input fromthe user.

When the first input is received, the processor 1300 may determine afirst binocular distance, i.e., a distance between the two eyes of theuser in a focal plane of the camera 1610, through an image sensor of thecamera 1610.

The processor 1300 may determine the first binocular distance, i.e., thedistance between the two eyes of the user in the focal plane of thecamera 1610, through photographing according to the first input. Thefirst binocular distance may be the distance between the two eyes of theuser in the focal plane of the camera 1610 at a point of time accordingto the first input. When the image sensor is located in the focal planeof the camera 1610, a distance between an image of a first eyeball andan image of a second eyeball of the user, which are formed on the imagesensor through the photographing according to the first input, may bethe first binocular distance. When the image sensor is located in thefocal plane of the camera 1610, the processor 1300 may determine thefirst binocular distance through information obtained from the imagesensor.

The processor 1300 may receive a second input for photographing the twoeyes of the user.

After receiving the first input, the processor 1300 may receive thesecond input used to photograph the two eyes of the user. For example,the processor 1300 may sequentially receive the first input and thesecond input requesting the device 100 to photograph the two eyes of theuser. For example, the processor 1300 may sequentially receive the firstand second inputs according to a touch input of the user received afterthe first input is received. As another example, the device 100 maysequentially receive the first and second inputs when a pre-setapplication is performed, without a particular input from the user.

When the second input is received, the processor 1300 may determine asecond binocular distance, i.e., a distance between the two eyes of theuser in the focal plane of the camera 1610, through the image sensor ofthe camera 1610.

The processor 1300 may determine the second binocular distance, i.e.,the distance between the two eyes of the user in the focal plane of thecamera 1610, through photographing according to the second input. Thesecond binocular distance may be the distance between the two eyes ofthe user on the focal plane of the camera 1610 at a point of timeaccording to the second input. When the image sensor is located in thefocal plane of the camera 1610, the distance between the image of thefirst eyeball and the image of the second eyeball of the user, which areformed on the image sensor through the photographing according to thesecond input, may be the second binocular distance. When the imagesensor is located on the focal plane of the camera 1610, the processor1300 may determine the second binocular distance through informationobtained from the image sensor.

The first binocular distance and the second binocular distance may bedifferent from each other. For example, the first and second binoculardistances may be different from each other when distances from the lensof the camera 1610 to the two eyes of the user at a photographing timeaccording to reception of the first input and at a photographing timeaccording to reception of the second input are different from eachother.

The processor 1300 may obtain a variation of the distance from thedevice 100 to the user between when the first input is received and whenthe second input is received. The processor 1300 may obtain informationabout motion of the device 100 from when the first input is received towhen the second input is received, and obtain the variation of thedistance based on the obtained information about the motion. A point oftime when the first input or the second input is received may includenot only a physical point of time when the first input or the secondinput is received, but also a point of time when the device 100 performsa pre-set operation according to the first input or the second input.For example, the processor 1300 may obtain the variation of the distancefrom the lens of the camera 1610 to the two eyes of the user between thephotographing time according to the reception of the first input and thephotographing time according to the reception of the second input.

The processor 1300 may obtain the variation of the distance by usinginformation obtained from a sensor. The sensor may obtain sensinginformation. For example, the sensor may obtain motion information ofthe device 100, and transmit the motion information to the processor1300.

The processor 1300 may determine a moving distance and a movingdirection of the device between the photographing time according to thereception of the first input and the photographing time according to thereception of the second input, by using the motion information obtainedby the sensor. The processor 1300 may determine the variation of thedistance from the lens of the camera 1610 to the two eyes of the userbetween the photographing time according to the reception of the firstinput and the photographing time according to the reception of thesecond input, based on the determined moving distance and the determinedmoving direction.

The processor 1300 may obtain a focal length of the camera 1610. Thefocal length of the camera 1610 may denote the distance from the lens ofthe camera 1610 to the focal plane of the camera 1610. Also, when theimage sensor is located in the focal plane, the focal length of thecamera 1610 or the focal length of the lens of the camera 1610 maydenote a distance from the lens of the camera 1610 to the image sensor.The focal length of the camera 1610 may be determined according tohardware characteristics of the camera 1610. For example, the focallength of the camera 1610 may be determined according to a refractiveindex of the lens of the camera 1610.

The processor 1300 may determine the distance between the device 100 andthe user based on the first binocular distance, the second binoculardistance, and the variation of the distance.

The processor 1300 may compare the first binocular distance and thesecond binocular distance to determine the distance from the lens of thecamera 1610 to the two eyes of the user at the photographing timeaccording to the reception of the first input and/or the distance fromthe lens of the camera 1610 to the two eyes of the user at thephotographing time according to the reception of the second input. Forexample, the processor 1300 may determine the distance from the lens ofthe camera 1610 to the two eyes of the user at the photographing timeaccording to the reception of the first input and/or the distance fromthe lens of the camera 1610 to the two eyes of the user at thephotographing time according to the reception of the second input, byusing the first binocular distance, the second binocular distance, thevariation of the distance, and the focal length of the camera 1610.

The sensing unit 1400 may detect a state of the device 100 or a statearound the device 100, and transmit the detected state to the processor1300.

The sensing unit 1400 may include at least one of a magnetic sensor1410, an acceleration sensor 1420, a temperature/humidity sensor 1430,an infrared sensor 1440, a gyroscope sensor 1450, a position sensor 1460such as global positioning system (GPS), an atmospheric sensor 1470, aproximity sensor 1480, and a red, green, blue (RGB) sensor 1490 such asan illuminance sensor, but components included in the sensing unit 1400are not limited thereto. Because functions of each sensor may beintuitively inferred by one of ordinary skill in the art based on itsname, details thereof are not described herein.

The communication unit 1500 may include at least one component enablingthe device 100 to communicate with an external device (not shown) or aserver (not shown). For example, the communication unit 1500 may includea short-range communication unit 1510, a mobile communication unit 1520,and a broadcast receiving unit 1530.

According to some embodiments, the short-range communication unit 1510may include a Bluetooth communication unit, a Bluetooth low energy (BLE)communication unit, a near-field communication (NFC) unit, a wirelesslocal area network (WLAN) (Wi-Fi) communication unit, a Zigbeecommunication unit, an infrared data association (IrDA) communicationunit, a Wi-Fi direct (WFD) communication unit, an ultra-wideband (UWB)communication unit, and an Ant+ communication unit, but componentsincluded in the short-range communication unit 1510 are not limitedthereto.

The mobile communication unit 1520 transmits and receives a wirelesssignal to and from at least one of a base station, an external terminal,and a server, on a mobile communication network. Here, a wireless signalmay include data having various formats according to transmission andreception of a voice call signal, a video telephone call signal, or atext/multimedia message.

The broadcast receiving unit 1530 receives a broadcast signal and/orbroadcast related information from an external source, through abroadcast channel. The broadcast channel may include a satellite channelor a terrestrial broadcasting channel. According to an embodiment, thedevice 100 may not include the broadcast receiving unit 1530.

Also, the communication unit 1500 may exchange information required todetermine the distance between the device 100 and the user, with theexternal device or the server.

The A/V input unit 1600 receives an audio signal or a video signal, andmay include the camera 1610 and a microphone 1620. The camera 1610 mayobtain an image frame of a still image or a moving image via the imagesensor in a video telephone mode or a photographing mode. An imagecaptured via the image sensor may be processed by the processor 1300 ora separate image processor (not shown).

An image frame processed by the camera 1610 may be stored in the memory1700 or transmitted to an external device through the communication unit1500. According to an embodiment of the device 100, there may be atleast two cameras 1610.

The microphone 1620 receives an external sound signal and processes theexternal sound signal to electric voice data. For example, themicrophone 1620 may receive a sound signal from an external device or anarrator. The microphone 1620 may use any one of various noise removingalgorithms to remove noise generated while receiving the external soundsignal.

The memory 1700 may store programs for processes and controls of theprocessor 1300, and may store data input to or output from the device100.

The memory 1700 may include at least one type of storage medium fromamong a flash memory, a hard disk, a multimedia card micro-type memory,a card-type memory (for example, a secure digital (SD) card or anextreme digital (XD) card), a random-access memory (RAM), a staticrandom-access memory (SRAM), a read-only memory (ROM), an electricallyerasable programmable read-only memory (EEPROM), a programmableread-only memory (PROM), a magnetic memory, a magnetic disk, and anoptical disk.

Programs stored in the memory 1700 may be classified into a plurality ofmodules based on functions, and may be classified into a user interface(UI) module 1710, a touch screen module 1720, and a notification module1730.

The UI module 1710 may provide a specialized UI or GUI linked to thedevice 100 according to applications. The touch screen module 1720 maydetect a touch gesture of a user on a touch screen, and transmitinformation about the touch gesture to the processor 1300. The touchscreen module 1720 according to an embodiment may recognize and analyzea touch code. The touch screen module 1720 may be configured as separatehardware including a controller.

Various sensors may be disposed inside or around the touch screen todetect a touch or a proximity touch on the touch screen. An example of asensor for detecting a touch on the touch screen includes a tactilesensor. The tactile sensor detects contact that can be felt by a personon a certain object. The tactile sensor may detect various types ofinformation, such as roughness of a contact surface, rigidness of acontact object, and temperature of a touch point.

Another example of a sensor for detecting a touch on the touch screenincludes a proximity sensor.

The proximity sensor detects existence of an object approaching or neara predetermined detection surface by using an electromagnetic fieldforce or infrared ray, without having to detect a mechanical contact.Examples of the proximity sensor include a transmission photoelectricsensor, a direct reflective type photoelectric sensor, a mirrorreflective type photoelectric sensor, a high-frequency oscillationproximity sensor, a capacitance-type proximity sensor, a magnetic-typeproximity sensor, and an infrared proximity sensor. Examples of a touchgesture of a user include tap, touch and hold, double-tap, drag,panning, flick, drag-and-drop, and swipe.

The notification module 1730 may generate a signal for notifying eventgeneration in the device 100. Examples of an event that is generated inthe device 100 include call signal reception, message reception, keysignal input, and schedule notification. The notification module 1730may output a notification signal in a video signal format through thedisplay unit 1210, in an audio signal format through the sound outputunit 1220, or in a vibration signal format through the vibration motor1230.

An embodiment of the present disclosure may also be realized in a formof a computer-readable recording medium, such as a program moduleexecuted by a computer. A computer-readable recording medium may be anarbitrary available medium accessible by a computer, and examplesthereof include all volatile and non-volatile media and separable andnon-separable media. Further, examples of the computer-readablerecording medium may include a computer storage medium and acommunication medium. Examples of the computer storage medium includeall volatile and non-volatile media and separable and non-separablemedia, which have been implemented by an arbitrary method or technology,for storing information such as computer-readable commands, datastructures, program modules, and other data. The communication mediumtypically include a computer-readable command, a data structure, aprogram module, other data of a modulated data signal, or anothertransmission mechanism, and an example thereof includes an arbitraryinformation transmission medium.

Also, in the present specification, a “unit” may be a hardwarecomponent, such as a processor or a circuit, and/or a software componentexecuted by a hardware component, such as a processor.

While the present disclosure has been particularly shown and describedwith reference to example embodiments thereof, it will be understood bythose of ordinary skill in the art that various changes in form anddetails may be made therein without departing from the spirit and scopeof the present disclosure as defined by the following claims. Hence, itwill be understood that the embodiments described above are not limitingthe scope of the disclosure. For example, each component described in asingle type may be executed in a distributed manner, and componentsdescribed distributed may also be executed in an integrated form.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims and their equivalents.

What is claimed is:
 1. A method by which a device determines a distance,the method comprising: determining a first binocular distance betweentwo eyes of a user by using a first image captured through a camera ofthe device; determining a second binocular distance between the two eyesof the user by using a second image captured through the camera;determining a distance from the camera to the two eyes of the user basedon the first binocular distance and the second binocular distance; andproviding an image to a display of the device, the image convertedaccording to the distance from the camera to the two eyes of the user.2. The method of claim 1, wherein the first binocular distance is thedistance between the two eyes of the user, and is recognized by an imagesensor of the camera when the first image is captured, and wherein thesecond binocular distance is the distance between the two eyes of theuser, and is recognized by the image sensor of the camera when thesecond image is captured.
 3. The method of claim 1, wherein the firstbinocular distance is the distance between the two eyes of the user in afocal plane of the camera when the first image is captured, and whereinthe second binocular distance is the distance between the two eyes ofthe user in the focal plane of the camera when the second image iscaptured.
 4. The method of claim 2, further comprising obtaining avariation of the distance from the camera to the two eyes of the user,wherein the variation of the distance is a variation between when thefirst image is captured and when the second image is captured, andwherein the determining of the distance from the camera to the two eyesof the user comprises determining the distance from the camera to thetwo eyes of the user based on the variation of the distance.
 5. Themethod of claim 4, wherein the obtaining of the variation of thedistance comprises: detecting a motion of the device from when the firstimage is captured to when the second image is captured; and obtainingthe variation of the distance based on the motion of the device.
 6. Themethod of claim 1, further comprising obtaining a focal length from alens of the camera to an image sensor of the camera, wherein thedetermining of the distance from the camera to the two eyes of the usercomprises determining the distance from the camera to the two eyes ofthe user based on the focal length.
 7. The method of claim 1, whereinthe providing of the image comprises obtaining the converted image basedon visual acuity information of the user stored in the device.
 8. Themethod of claim 1, further comprising: predicting a size of an eyeballof the user based on information about the user stored in the device;determining the size of the eyeball of the user located in a focal planeof the camera; and determining a distance from the camera to an eyeballof the user based on the predicted eyeball size and the determinedeyeball size.
 9. The method of claim 8, further comprising obtaining afocal length from a lens of the camera to an image sensor of the camera,wherein the determining of the distance from the camera to the eyeballof the user comprises determining the distance from the camera to theeyeball of the user based on the focal length.
 10. The method of claim1, further comprising: determining a degree of tilt of the device;determining a line of sight of the user; and updating the distance fromthe camera to the two eyes of the user based on an angle determinedaccording to the degree of tilt of the device and the line of sight ofthe user.
 11. A device for determining a distance, the devicecomprising: a camera configured to photograph two eyes of a user; and aprocessor configured to: determine a first binocular distance betweenthe two eyes of the user by using a first image captured through thecamera, determine a second binocular distance between the two eyes ofthe user by using a second image captured through the camera, determinea distance from the camera to the two eyes of the user based on thefirst binocular distance and the second binocular distance, and providean image to a display of the device, the image converted according tothe distance from the camera to the two eyes of the user.
 12. The deviceof claim 11, wherein the first binocular distance is the distancebetween the two eyes of the user in the first image formed on an imagesensor of the camera when the first image is captured, and wherein thesecond binocular distance is the distance between the two eyes of theuser in the second image formed on the image sensor of the camera whenthe second image is captured.
 13. The device of claim 11, wherein thefirst binocular distance is the distance between the two eyes of theuser in a focal plane of the camera when the first image is captured,and wherein the second binocular distance is the distance between thetwo eyes of the user in the focal plane of the camera when the secondimage is captured.
 14. The device of claim 12, wherein the processor isfurther configured to obtain a variation of the distance from the camerato the two eyes of the user, from when the first image is captured towhen the second image is captured, and wherein the distance from thecamera to the two eyes of the user is determined based on the variationof the distance.
 15. The device of claim 14, further comprising a motionsensor configured to detect a motion of the device, wherein theprocessor is further configured to: detect the motion of the device fromwhen the first image is captured to when the second image is captured,and obtain the variation of the distance based on the motion of thedevice.
 16. The device of claim 11, wherein the processor is furtherconfigured to: obtain a focal length from a lens of the camera to animage sensor of the camera, and determine the distance from the camerato the two eyes of the user based on the focal length.
 17. The device ofclaim 11, wherein the processor is further configured to obtain theconverted image based on visual acuity information of the user stored inthe device.
 18. The device of claim 11, wherein the processor is furtherconfigured to: predict a size of an eyeball of the user based oninformation about the user stored in the device, determine the size ofthe eyeball of the user in a focal plane of the camera, and determine adistance from the camera to the eyeball of the user based on thepredicted eyeball size and the determined eyeball size.
 19. The deviceof claim 18, wherein the processor is further configured to: obtain afocal length from a lens of the camera to an image sensor of the camera,and determine the distance from the camera to the eyeball of the userbased on the focal length.
 20. A non-transitory computer-readablerecording medium having recorded thereon a program which, when executedby a computer, performs the method of claim 1.